Data Description:

You are provided with a training set and a test set of images of 3 classes. Each image has a filename that is its unique id. The dataset comprises 3 classes: COVID-19, Viral Pneumonia, and Normal. The goal of the projectis to create a classifier capable of determining the class of the X-ray image.

Dataset:

The project isinspiredfrom a dataset from Kaggle.Note: For project purposes, the dataset is modified according to the learning requirements.You are provided with the following files:

• testimage.npy
• testLabels.csv
• trainimage.npy
• trainLabels.csv

Context:

• Can you differentiate an X-ray image of a normal person from an unhealthy one?
• The ability to do so effectively can mean better diagnosis.

#Mounting the drive 
from google.colab import drive
drive.mount('/content/gdrive')

Mounted at /content/gdrive

#Importing the required libraries 
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cv2 as cv
import os
import seaborn as sns # for data visualization 
import tensorflow as tf
import keras
from tensorflow.keras.models import Sequential #sequential api for sequential model 
from tensorflow.keras.layers import Dense, Dropout, Flatten #importing different layers 
from tensorflow.keras.layers import Conv2D, MaxPooling2D, BatchNormalization, Activation, Input, LeakyReLU,Activation
from tensorflow.keras import backend as K
from tensorflow.keras.utils import to_categorical #to perform one-hot encoding 
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D
from tensorflow.keras.optimizers import RMSprop,Adam #optimiers for optimizing the model
from keras.callbacks import EarlyStopping  #regularization method to prevent the overfitting
from keras.callbacks import ModelCheckpoint
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras import losses, optimizers

Reading the Dataset

#Load the image files datasets  
images_test=np.load('/content/gdrive/MyDrive/covid-19 dataset/testimage.npy')
images_train=np.load('/content/gdrive/MyDrive/covid-19 dataset/trainimage.npy')

# Load the labels files of dataset
label_test = pd.read_csv('/content/gdrive/MyDrive/covid-19 dataset/testLabels.csv')
label_train = pd.read_csv('/content/gdrive/MyDrive/covid-19 dataset/trainLabels.csv')

Overview of the dataset

#Checking the shape of all the datasets
print(images_train.shape)
print(label_train.shape)
print(images_test.shape)
print(label_test.shape)

(251, 128, 128, 3)
(251, 1)
(66, 128, 128, 3)
(66, 1)

• There are 251 RGB images in the training set having the size 128X128 with 3 channels .
• There are 66 RGB images in the testing set having the size 128X128 with 3 channels

# Show few images in the training set 
for i in range(8):
    plt.subplot(2, 4, i + 1)
    plt.imshow(images_train[i])

label_train.head()#first 5 entries of the label training set 

	Label
0	Viral Pneumonia
1	Viral Pneumonia
2	Viral Pneumonia
3	Viral Pneumonia
4	Viral Pneumonia

label_test.head()# first 5 entries of the label testing set 

	Label
0	Viral Pneumonia
1	Viral Pneumonia
2	Viral Pneumonia
3	Viral Pneumonia
4	Viral Pneumonia

label_train['Label'].value_counts()# checking the count of different labels in the training set 

Covid              111
Normal              70
Viral Pneumonia     70
Name: Label, dtype: int64

• 111 Covid X-rays
• 70 Normal
• 70 Viral Pneumonia
• Equal number of normal and Viral Pneumonia cases in training set

label_test['Label'].value_counts()

Covid              26
Normal             20
Viral Pneumonia    20
Name: Label, dtype: int64

• 26 Covid X-rays
• 20 Normal
• 20 Viral Pneumonia
• Equal number of normal and Viral Pneumonia cases in testing set

label_train['Label'].unique()# checking the different labels 

array(['Viral Pneumonia', 'Covid', 'Normal'], dtype=object)

label_test['Label'].unique()# checking the different labels 

array(['Viral Pneumonia', 'Covid', 'Normal'], dtype=object)

sns.countplot(label_train['Label'])# To see the distibution of various labels 
plt.xticks(rotation='vertical')

/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning

(array([0, 1, 2]), <a list of 3 Text major ticklabel objects>)

We see that most of the labels in the training set are of covid patients followed by Viral Pneumonia and normal having the same amonut of patients

Mean Images for all the labels

def find_mean_img(full_mat):#function to get the mean images of all the labels
    # calculate the average
    mean_img = np.mean(full_mat, axis = 0)
    # reshape it back to a matrix
    mean_img = mean_img.reshape((150,150))
  
    return mean_img

CATEGORIES=label_train['Label'].unique()
d={ i:[] for i in CATEGORIES}

for i in label_train.index:
  gray = cv.cvtColor(images_train[i], cv.COLOR_BGR2GRAY)
  gray = cv.resize(gray,(150,150))
  d[label_train['Label'][i]].append(gray)

l=[]
for i in d.keys():
  l.append(find_mean_img(d[i]))

plt.subplots(figsize=(12,12))
for i in range(len(l)):
    plt.subplot(3,4,i + 1,title='Average '+list(d.keys())[i])
    plt.imshow(l[i],cmap='gray')
    plt.axis('off')

Data Preprocessing

Gaussian Blurring

def gaussImage(inImageArray):#function for gaussian blurr
    
    gaussImageArray=[]
    
    for i in inImageArray: 
        gaussImage = cv.GaussianBlur(i,(5,5),0)
        gaussImageArray.append(gaussImage)
        
    gaussImageArray=np.asarray(gaussImageArray)
    return gaussImageArray  

Normalizing images

def normalizeImage(inImageArray):#function for normalizing 
    
    normImageArray=[]
    
    for i in inImageArray:
        
        norm_img = np.zeros((128,128))
        final_img = cv.normalize(i,  norm_img, 0, 255, cv.NORM_MINMAX)
        normImageArray.append(final_img)
    
    normImageArray=np.asarray(normImageArray)
    return normImageArray


def showImages(fromImage,imageArray,gray):# functin to display images
    
    fig, axs = plt.subplots(nrows=5,ncols=5, sharex=True, figsize=(16, 16))
    imgCount=fromImage
    
    for x in range(5):
        
        imgCount=imgCount+x
        
        for y in range(5):
            
            imgCount=imgCount+y
            
            if (gray):
                axs[x][y].imshow(imageArray[imgCount],cmap='gray', vmin=0, vmax=255)
            else:
                axs[x][y].imshow(imageArray[imgCount])
                

def displayImageSideBySide(fromImage,origImageArray,normImageArray,gaussImageArray,gray):#function to display images side by side 
    
    fig, axs = plt.subplots(nrows=1,ncols=3, sharex=True, figsize=(25, 5))
    imgCount=fromImage
    if (gray):
        axs[0].imshow(origImageArray[imgCount],cmap='gray')
        axs[1].imshow(normImageArray[imgCount],cmap='gray')
        axs[2].imshow(gaussImageArray[imgCount],cmap='gray')
    else:
        axs[0].imshow(origImageArray[imgCount])
        axs[1].imshow(normImageArray[imgCount])
        axs[2].imshow(gaussImageArray[imgCount])



showImages(0,images_train,False)# seeing  few images in the training set 

showImages(1,images_test,False)# viewing few images in the testing set

showImages(100,images_train,False)# checking few images in the training set 

showImages(150,images_train,False)# checking few images in the training set 

normalTrainImg = normalizeImage(images_train)
gaussTrainImg = gaussImage(normalTrainImg) 
displayImageSideBySide(0,images_train,normalTrainImg,gaussTrainImg,False)# displaying training images before treatment,after normalized and after gaussian blurring 

displayImageSideBySide(100,images_train,normalTrainImg,gaussTrainImg,False)# displaying training images before treatment,after normalized and after gaussian blurring 

normalTestImg = normalizeImage(images_test)
gaussTestImg = gaussImage(normalTestImg) 
displayImageSideBySide(0,images_test,normalTestImg,gaussTestImg,False)# displaying testing images before treatment,after normalized and after gaussian blurring 

from skimage.color import rgb2gray
def toGrayScale(inImageArray):# function to convert to greyscale
    
    gtrainImg=[]
    
    for i in inImageArray: 
      img = rgb2gray(i)

      gtrainImg.append(img)
    
    gtrainImg=np.asarray(gtrainImg)
        
    return gtrainImg

def noiseReduction(inImageArray):# function for noise reduction 
    
    dtrainImg=[]
    inImageArray = inImageArray.reshape(inImageArray.shape[0], 128, 128, 3)
    inImageArray = inImageArray.astype('float32')
    
    for i in inImageArray:
        image = i
        image=(image*255).astype(np.uint8)
        
        img=cv.fastNlMeansDenoising(image)
        
        dtrainImg.append(img)
    dtrainImg=np.asarray(dtrainImg)
    
    return dtrainImg

def completeDenoise(inImageArray):# function for complete noise reduction
  gtrainImg=[]
  for i in inImageArray:
    gray = cv.cvtColor(i,cv.COLOR_BGR2GRAY)
    img=gray
    images_train.append(img)

  gtrainImg=np.asarray(gtrainImg)
  gtrainImg=noiseReduction(gtrainImg) #denoise images
    
    
  return gtrainImg

gtrainImg=toGrayScale(images_train)

gnormalTrainImg = normalizeImage(gtrainImg)
ggaussTrainImg = gaussImage(gnormalTrainImg)

displayImageSideBySide(100,gtrainImg,gnormalTrainImg,ggaussTrainImg,True)# displaying the greyscale images ,normalized images and the gaussian blurred images 

denoisedImg=noiseReduction(normalTrainImg)
displayImageSideBySide(110,images_train,normalTrainImg,denoisedImg,True)# displaying the images before treatment ,normalized images and the denoised images 

showImages(150,denoisedImg,True)#displaying the denoised images 

denoisedtestImg=noiseReduction(normalTestImg)
displayImageSideBySide(10,images_test,normalTestImg,denoisedtestImg,True)# displaying the normal test images ,normalized and denoised test images 

Making the data compatible

from tensorflow.keras import utils
label_train["Label"] = label_train["Label"].astype('category')
label_train["LabelID"] = label_train["Label"].cat.codes

Y_train_initial = label_train['LabelID'].values
Y_train_lab = utils.to_categorical(Y_train_initial, num_classes=12)

print("Shape of y_train:", Y_train_lab.shape)
print("One value of y_train:", Y_train_lab[0])

label_train.Label.value_counts().plot(kind='bar')

labels_train=label_train.drop_duplicates(subset ="Label") 

labels_train=label_train.sort_values(by=['LabelID'], ascending=True)

labelList = labels_train["Label"].tolist()
print(labelList)

Shape of y_train: (251, 12)
One value of y_train: [0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
['Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia']

We see that the size of y_train is (251,12).Covid is the most common label forllowed by Viral pneumonia and normal

label_test["Label"] = label_test["Label"].astype('category')
label_test["LabelID"] = label_test["Label"].cat.codes

Y_test_initial = label_test['LabelID'].values
Y_test_lab = utils.to_categorical(Y_test_initial, num_classes=12)

print("Shape of y_test:", Y_test_lab.shape)
print("One value of y_test:", Y_test_lab[0])

label_test.Label.value_counts().plot(kind='bar')

labels_test=label_test.drop_duplicates(subset ="Label") 

labels_test=label_test.sort_values(by=['LabelID'], ascending=True)

labeltestList = labels_test["Label"].tolist()
print(labeltestList)

Shape of y_test: (66, 12)
One value of y_test: [0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
['Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Covid', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Normal', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia', 'Viral Pneumonia']

We see that the size of y_train is (66,12).Covid is the most common label forllowed by Viral pneumonia and normal

Splitting the training set to training and validaton set

from sklearn.model_selection import train_test_split
X_train=denoisedImg
X_train2, X_val, Y_train2, Y_val = train_test_split(X_train, Y_train_lab, test_size=0.5, random_state=42,
                                                  stratify = Y_train_lab )

X_test=denoisedtestImg
Y_test=Y_test_lab

#Checking the size of train, test and validation 
print("X_train shape:",X_train2.shape)
print("Y_train shape:",Y_train2.shape)

print("X_val shape:",X_val.shape)
print("Y_val shape:",Y_val.shape)

print("X_test shape:",X_test.shape)
print("Y_test shape:",Y_test.shape)

#Converting train, test and validation to float 
X_train2 = X_train2.astype('float32')
X_test = X_test.astype('float32')
X_val = X_val.astype('float32')

# X_train /= 255
# X_test /=255
# X_val /= 255

#Displaying the shape,min and max
print("X_train shape:", X_train2.shape)
print("Images in X_train:", X_train2.shape[0])
print("Images in X_val:", X_val.shape[0])
print("Images in X_test:", X_test.shape[0])
print("Max value in X_train:", X_train2.max())
print("Min value in X_train:", X_train2.min())

X_train shape: (125, 128, 128, 3)
Y_train shape: (125, 12)
X_val shape: (126, 128, 128, 3)
Y_val shape: (126, 12)
X_test shape: (66, 128, 128, 3)
Y_test shape: (66, 12)
X_train shape: (125, 128, 128, 3)
Images in X_train: 125
Images in X_val: 126
Images in X_test: 66
Max value in X_train: 255.0
Min value in X_train: 0.0

Reshaping the train,test and validation set

X_test_flat=X_test

print("Reshaping the X_train, X_val")
X_train2 = X_train2.reshape(X_train2.shape[0], 128, 128, 3)
X_test = X_test.reshape(X_test.shape[0], 128, 128, 3)
X_val = X_val.reshape(X_val.shape[0], 128, 128, 3)

print("X_train shape:",X_train2.shape)
print("X_val shape:",X_val.shape)
print("X_test shape:",X_test.shape)

Reshaping the X_train, X_val
X_train shape: (125, 128, 128, 3)
X_val shape: (126, 128, 128, 3)
X_test shape: (66, 128, 128, 3)

Building the model

def construct_model():
    
    model = Sequential() 
    
    model.add(Conv2D(filters=64, kernel_size=(3, 3), input_shape=(128, 128, 3), activation='relu'))
    model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
    model.add(MaxPool2D((2, 2)))

    model.add(Flatten())

    model.add(Dense(128, activation='relu'))

    model.add(Dense(12, activation='softmax'))
    
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    
    return model

def sconstruct_model():
    smodel = Sequential()    
    smodel.add(Conv2D(filters=64, kernel_size=(3, 3), input_shape=(128, 128, 3), activation='relu'))
    smodel.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
    smodel.add(MaxPool2D((2, 2)))
    
    smodel.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
    smodel.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
    smodel.add(MaxPool2D((2, 2)))

    smodel.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
    smodel.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
    smodel.add(MaxPool2D((2, 2)))
    
    smodel.add(Flatten())

    smodel.add(Dense(128, activation='relu'))

    smodel.add(Dense(12, activation='softmax'))
    #optimizer = Adam(lr=0.001)
    
    smodel.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    #model.summary()
    return smodel

def cconstruct_model(learningRate):
    smodel = Sequential()    
    
    smodel.add(Conv2D(filters=32, kernel_size=(3, 3), input_shape=(128, 128, 3), activation='relu'))
    smodel.add(Conv2D(filters=32, kernel_size=(3, 3), activation='relu'))
    smodel.add(MaxPool2D((2, 2)))
        
    smodel.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
    smodel.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
    smodel.add(MaxPool2D((2, 2)))
    
    smodel.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
    smodel.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))
    smodel.add(MaxPool2D((2, 2)))

    smodel.add(Flatten())

    smodel.add(Dense(128, activation='relu'))
    smodel.add(Dense(128, activation='relu'))

    smodel.add(Dense(12, activation='softmax'))
    optimizer = Adam(lr=learningRate)
    
    smodel.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])
    smodel.summary()
    return smodel

model=cconstruct_model(0.001)

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 126, 126, 32)      896       
                                                                 
 conv2d_1 (Conv2D)           (None, 124, 124, 32)      9248      
                                                                 
 max_pooling2d (MaxPooling2D  (None, 62, 62, 32)       0         
 )                                                               
                                                                 
 conv2d_2 (Conv2D)           (None, 60, 60, 64)        18496     
                                                                 
 conv2d_3 (Conv2D)           (None, 58, 58, 64)        36928     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 29, 29, 64)       0         
 2D)                                                             
                                                                 
 conv2d_4 (Conv2D)           (None, 27, 27, 128)       73856     
                                                                 
 conv2d_5 (Conv2D)           (None, 25, 25, 128)       147584    
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 12, 12, 128)      0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 18432)             0         
                                                                 
 dense (Dense)               (None, 128)               2359424   
                                                                 
 dense_1 (Dense)             (None, 128)               16512     
                                                                 
 dense_2 (Dense)             (None, 12)                1548      
                                                                 
=================================================================
Total params: 2,664,492
Trainable params: 2,664,492
Non-trainable params: 0
_________________________________________________________________

/usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/adam.py:105: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(Adam, self).__init__(name, **kwargs)

callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=10, min_delta=0.005)

checkpoint = tf.keras.callbacks.ModelCheckpoint('model.h5',monitor='accuracy',verbose=0, save_best_only=True)
checkpoint2 = tf.keras.callbacks.ModelCheckpoint('model2.h5',monitor='accuracy',verbose=0, save_best_only=True)

# Use earlystopping
history=model.fit( x=X_train2, y=Y_train2,epochs=15,validation_data=(X_val, Y_val), callbacks=[callback,checkpoint],verbose=1)

Epoch 1/15
4/4 [==============================] - 16s 715ms/step - loss: 16.5128 - accuracy: 0.2960 - val_loss: 1.4248 - val_accuracy: 0.2778
Epoch 2/15
4/4 [==============================] - 0s 129ms/step - loss: 1.5037 - accuracy: 0.2880 - val_loss: 1.1508 - val_accuracy: 0.4048
Epoch 3/15
4/4 [==============================] - 1s 167ms/step - loss: 0.9839 - accuracy: 0.5920 - val_loss: 1.2191 - val_accuracy: 0.6508
Epoch 4/15
4/4 [==============================] - 1s 175ms/step - loss: 0.9258 - accuracy: 0.6000 - val_loss: 0.7143 - val_accuracy: 0.5714
Epoch 5/15
4/4 [==============================] - 1s 165ms/step - loss: 0.6836 - accuracy: 0.6640 - val_loss: 0.5966 - val_accuracy: 0.7540
Epoch 6/15
4/4 [==============================] - 1s 184ms/step - loss: 0.4564 - accuracy: 0.7840 - val_loss: 0.6177 - val_accuracy: 0.7302
Epoch 7/15
4/4 [==============================] - 0s 123ms/step - loss: 0.4757 - accuracy: 0.7360 - val_loss: 0.4371 - val_accuracy: 0.8492
Epoch 8/15
4/4 [==============================] - 1s 164ms/step - loss: 0.4027 - accuracy: 0.8720 - val_loss: 0.4064 - val_accuracy: 0.8571
Epoch 9/15
4/4 [==============================] - 1s 165ms/step - loss: 0.2239 - accuracy: 0.9360 - val_loss: 0.4707 - val_accuracy: 0.8175
Epoch 10/15
4/4 [==============================] - 1s 164ms/step - loss: 0.1859 - accuracy: 0.9440 - val_loss: 0.3691 - val_accuracy: 0.8571
Epoch 11/15
4/4 [==============================] - 0s 132ms/step - loss: 0.1769 - accuracy: 0.9360 - val_loss: 0.4413 - val_accuracy: 0.8571
Epoch 12/15
4/4 [==============================] - 0s 122ms/step - loss: 0.1236 - accuracy: 0.9440 - val_loss: 0.3833 - val_accuracy: 0.8571
Epoch 13/15
4/4 [==============================] - 1s 171ms/step - loss: 0.0677 - accuracy: 0.9840 - val_loss: 0.4674 - val_accuracy: 0.8413
Epoch 14/15
4/4 [==============================] - 0s 123ms/step - loss: 0.0470 - accuracy: 0.9840 - val_loss: 0.4107 - val_accuracy: 0.8651
Epoch 15/15
4/4 [==============================] - 0s 121ms/step - loss: 0.0419 - accuracy: 0.9760 - val_loss: 0.4881 - val_accuracy: 0.8413

plt.plot(history.history['accuracy'], label='accuracy')
plt.plot(history.history['val_accuracy'], label = 'val_accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.ylim([0.5, 1])
plt.legend(loc='lower right');

We see that the model is generalizing well to the validation set.

# Evaluating the model.

score = model.evaluate(X_test, Y_test, verbose=0, batch_size = 10)
print('Test loss:', score[0])
print('Test accuracy:', score[1])

Test loss: 0.4454047679901123
Test accuracy: 0.8636363744735718

We have got a good accuracy on the testing set of 86.36% and the model has generalized well with the validation set

Visualizing the Predictions

def Prediction(Model):
    print("Test accuracy",model.evaluate(X_test, Y_test))
    print("Train accuracy",model.evaluate(X_train, Y_train))
    print("Validation accuracy",model.evaluate(X_val, Y_val))
    # PREDICTIONS
Y_pred = model.predict(X_test)
Y_predicted = np.argmax(Y_pred, axis = 1) 
Y_true = np.argmax(Y_test, axis = 1) 

X_test_image = X_test.reshape(X_test.shape[0], 128, 128,3)

def showImage(position):
    print("Predicted",Y_pred[position].argmax(),"Actual ",Y_test[position].argmax())
    plt.imshow((X_test_image[position]*255).astype(np.uint8),cmap='gray')

showImage(2)

Predicted 2 Actual  2

showImage(3)

Predicted 0 Actual  2

showImage(33)

Predicted 0 Actual  0

showImage(36)

Predicted 0 Actual  0

showImage(59)

Predicted 2 Actual  1

from tensorflow.keras.models import load_model
from sklearn.metrics import confusion_matrix

# Predict the values from the validation dataset
Y_pred = model.predict(X_test)
# Convert predictions classes to one hot vectors 
result = np.argmax(Y_pred, axis=1)
# Convert validation observations to one hot vectors
Y_true = np.argmax(Y_test, axis=1)




conf_mat = confusion_matrix(Y_true, result)

df_cm = pd.DataFrame(conf_mat, index = [i for i in range(0,3)],
                  columns = [i for i in range(0,3 )])
plt.figure(figsize = (10,7))
sns.heatmap(df_cm, annot=True, fmt='g');


model = load_model('model.h5')
Y_pred= confusionMatrix(model)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-52-2858e46b47a6> in <module>()
     21 
     22 model = load_model('model.h5')
---> 23 Y_pred= confusionMatrix(model)

NameError: name 'confusionMatrix' is not defined

We see that we have 9 entries that have been classified incorrectly however the majority have been classified correctly

Reducing Overfitting

from tensorflow.keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import ReduceLROnPlateau

learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy', 
                                            patience=3, 
                                            verbose=1, 
                                            factor=0.5, 
                                            min_lr=0.00001)
epochs = 15
batch_size = 20

datagen = ImageDataGenerator(  
        rotation_range=90,  # randomly rotate images in the range
        zoom_range = 0.1, # Randomly zoom image 
        width_shift_range=0.1,  # randomly shift images horizontally
        height_shift_range=0.1,  # randomly shift images vertically 
        horizontal_flip=True,  # randomly flip images horizontally
        vertical_flip=True  # randomly flip images vertically
    )  
datagen.fit(X_train2)

# Set the CNN model 
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, GlobalMaxPooling2D
batch_size=None
model1 = Sequential()

model1.add(Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', 
                 activation ='relu', batch_input_shape = (batch_size,128, 128, 3)))


model1.add(Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', 
                 activation ='relu'))
model1.add(MaxPool2D(pool_size=(2,2)))
model1.add(Dropout(0.2))


model1.add(Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', 
                 activation ='relu'))
model1.add(Conv2D(filters = 64, kernel_size = (3,3),padding = 'same', 
                 activation ='relu'))
model1.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model1.add(Dropout(0.3))

model1.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same', 
                 activation ='relu'))
model1.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same', 
                 activation ='relu'))
model1.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model1.add(Dropout(0.4))



model1.add(GlobalMaxPooling2D())
model1.add(Dense(128, activation = "relu"))
model1.add(Dropout(0.5))
model1.add(Dense(12, activation = "softmax"))
model1.summary()

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_6 (Conv2D)           (None, 128, 128, 32)      2432      
                                                                 
 conv2d_7 (Conv2D)           (None, 128, 128, 32)      25632     
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 64, 64, 32)       0         
 2D)                                                             
                                                                 
 dropout (Dropout)           (None, 64, 64, 32)        0         
                                                                 
 conv2d_8 (Conv2D)           (None, 64, 64, 64)        18496     
                                                                 
 conv2d_9 (Conv2D)           (None, 64, 64, 64)        36928     
                                                                 
 max_pooling2d_4 (MaxPooling  (None, 32, 32, 64)       0         
 2D)                                                             
                                                                 
 dropout_1 (Dropout)         (None, 32, 32, 64)        0         
                                                                 
 conv2d_10 (Conv2D)          (None, 32, 32, 128)       73856     
                                                                 
 conv2d_11 (Conv2D)          (None, 32, 32, 128)       147584    
                                                                 
 max_pooling2d_5 (MaxPooling  (None, 16, 16, 128)      0         
 2D)                                                             
                                                                 
 dropout_2 (Dropout)         (None, 16, 16, 128)       0         
                                                                 
 global_max_pooling2d (Globa  (None, 128)              0         
 lMaxPooling2D)                                                  
                                                                 
 dense_3 (Dense)             (None, 128)               16512     
                                                                 
 dropout_3 (Dropout)         (None, 128)               0         
                                                                 
 dense_4 (Dense)             (None, 12)                1548      
                                                                 
=================================================================
Total params: 322,988
Trainable params: 322,988
Non-trainable params: 0
_________________________________________________________________

#Defining thr optimizer and loss function 
optimizer = RMSprop(lr=0.001, rho=0.9, epsilon=1e-08, decay=0.0)
model1.compile(optimizer = optimizer, loss = "categorical_crossentropy", metrics = ["accuracy"])

/usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/rmsprop.py:130: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(RMSprop, self).__init__(name, **kwargs)

history1=model.fit(datagen.flow(X_train2, Y_train2), epochs=15, validation_data=(X_val, Y_val), callbacks=[callback,checkpoint2],verbose=1)

Epoch 1/15
4/4 [==============================] - 2s 350ms/step - loss: 1.9808 - accuracy: 0.5280 - val_loss: 0.6342 - val_accuracy: 0.7857
Epoch 2/15
4/4 [==============================] - 1s 228ms/step - loss: 1.1778 - accuracy: 0.4800 - val_loss: 0.8192 - val_accuracy: 0.6905
Epoch 3/15
4/4 [==============================] - 1s 224ms/step - loss: 1.0968 - accuracy: 0.4320 - val_loss: 0.8075 - val_accuracy: 0.6984
Epoch 4/15
4/4 [==============================] - 1s 215ms/step - loss: 1.0205 - accuracy: 0.5120 - val_loss: 0.7190 - val_accuracy: 0.6825
Epoch 5/15
4/4 [==============================] - 1s 214ms/step - loss: 0.9358 - accuracy: 0.5200 - val_loss: 0.7371 - val_accuracy: 0.7857
Epoch 6/15
4/4 [==============================] - 1s 267ms/step - loss: 0.8541 - accuracy: 0.6080 - val_loss: 0.6423 - val_accuracy: 0.8175
Epoch 7/15
4/4 [==============================] - 1s 209ms/step - loss: 0.8884 - accuracy: 0.5520 - val_loss: 0.6722 - val_accuracy: 0.7302
Epoch 8/15
4/4 [==============================] - 1s 249ms/step - loss: 0.8182 - accuracy: 0.6240 - val_loss: 0.8043 - val_accuracy: 0.5397
Epoch 9/15
4/4 [==============================] - 1s 204ms/step - loss: 0.7873 - accuracy: 0.5840 - val_loss: 0.7790 - val_accuracy: 0.7063
Epoch 10/15
4/4 [==============================] - 1s 242ms/step - loss: 0.7383 - accuracy: 0.6320 - val_loss: 0.6972 - val_accuracy: 0.7222
Epoch 11/15
4/4 [==============================] - 1s 249ms/step - loss: 0.7921 - accuracy: 0.6880 - val_loss: 0.8531 - val_accuracy: 0.6746
Epoch 12/15
4/4 [==============================] - 1s 246ms/step - loss: 0.6284 - accuracy: 0.7280 - val_loss: 0.6741 - val_accuracy: 0.7302
Epoch 13/15
4/4 [==============================] - 1s 202ms/step - loss: 0.7229 - accuracy: 0.6960 - val_loss: 0.7756 - val_accuracy: 0.7381
Epoch 14/15
4/4 [==============================] - 1s 252ms/step - loss: 0.4767 - accuracy: 0.8080 - val_loss: 1.2007 - val_accuracy: 0.6667
Epoch 15/15
4/4 [==============================] - 1s 205ms/step - loss: 0.5948 - accuracy: 0.7600 - val_loss: 0.6004 - val_accuracy: 0.8095

plt.plot(history1.history['accuracy'], label='accuracy')
plt.plot(history1.history['val_accuracy'], label = 'val_accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.ylim([0.5, 1])
plt.legend(loc='lower right');

The model isnt generalising well with the validation set

# Evaluate the model.

score1 = model1.evaluate(X_test, Y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])

Test loss: 0.4454047679901123
Test accuracy: 0.8636363744735718

The accuracy remains the same in the second model

Conclusion:

• Since we have seen a much better regulariztion with the first model and not much of a difference in the accuracy ,it is preferred
• The model accuracy becomes constant after the 7th epoch, and reducing learning rate doesnt works well at this time.
• The model gives us a accuracy of 86.36% on a dataset having 66 entries(missclassifying 9 entries )